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ABSTRACT 

School districts receiving Chapter 1 funds are to 
conduct a study of sustained effects once every 3 years and to use 
the results for planning future Chapter 1 programs. As part of this 
effort, the Columbus (Ohio) public schools conducted 
pretest-to-post test , posttest-to-sustained effect, and 
pretest-to-sustained effect studies of 4,031 students in grades 2 
through 7 who participated in Chapter 1 reading and mathematics 
programs in the 1992-93 school year. Data for 16 analysis groups 
(reading and m?^thematics at different levels were collected in April 
of 1992 (pretest), 1993 (posttest), and 1994 (sustained effects). 
Those who exited the Chapter 1 program had the highest postte^^t 
scores on average, but lost a substantial portion of this gain in the 
posttes^-to-sustained effect period. At least 50 percent of students 
in 14 of the 16 study groups had positive pretest-to-sustained effect 
change values, higher than those of "evaluation" (comparison) groups. 
The results agree with the findings of the similar sustained effects 
study for 1989-91. Twenty-two figures and 24 tables are included. 
(Contains 3 references . ) (SLD) 
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Elementary and Secondary Education Act - Chapter 1 



REPORT OF THE CHAPTER 1 
SUSTAINED EFFECTS STUDY 
1992-94 



ABSTRACT 



Purpose of thfl Study 



According to ESEA Chapter 1 guidelines, school districts receiving Chapter 1 funds are to conduct a 
study of sustained effects once every three years and use the results in planning future Chapter 1 
programs. To this end, this study addresses three researcti questions related to Normal Curve Equivalent 
(NCE) change values: a) What is the change from pretest to posttest?; b) What is the change from 
posttest to sustained effect?; and, c) What is the change from pretest to sustained effect? 

Method 



A Spnng - Spring - Spring testing design was adopted. Data were collected in April of three 
successive years: 1992 (pretest), 1993 (posttest), 1994 (sustained effects). Five eligibii;iy criteria were 
used to judge whether a pupil would become a part of the dataset used for analysis purposes. Each pupil 
had to: a) participate in a Chapter 1 reading/math program in Columbus Public Schools during the 1992-93 
school year; b) be in grade 2-7; c) be in the 1992-93 evaluation sample; d) be promoted to the next grade 
for the 1993-94 school year; and, e) have test scores for ail three Spring measurements. Data from these 
measurements were matched by student number; pupils not meeting all five criteria were eliminated from 
the study. 



Data were partitioned by skill area (Reading-Basic; Reading-Advanced; Math-Basic; Math- 
Advanced), and by group-of-interest (exited program after one year; in the evaluation sample the second 
year; served but not in 'he evaluation sample the second year; total). Thus, IS analysis groups were 



BssuHs. 



From the 1992-93 Chapter 1 data file. 4.031 pupil records met the first three eligibility criteria and were 
considered for this study: 1.821 Reading records and 2.210 Math records. After selection based on 
eligibility cntena four and five, and further partitioning by group-of interest, these counts were observed- 
Reading Basic=1 .425; Reading Advanced=1, 466; Math Basic= 1,751; and Math Advanced=1 ,790. 

Pupils who exited the Chapter 1 program had the highest posttest scores, on average. Pupils in the 
evaluation sample a d served subsets were lower at posttesting than pupils in the exited group, and were 
quite similar in tem^s of change values. Served pupils did posttest-wsn average-higher than evaluation- 
sample pupils. Exited pupils tended to lose more pretest-to-posttest change than did other pupils when 
posttest-to-sustained effect change was evaluated. Evaluation sample (Reading-Basic) pupils, on averaae 
actually had lower posttest scores than pretest scores. ' 

Exited pupils in the Math-Basic subset tost more than 45% of the positive pretest-to-posttest change 
when posttest-sustained effect change was evaluated. When pretest-to-sustained effect change was 
evaluated, 14 of the 16 study groups showed positive change. Served pupils in Math Basic and Math 
Advanced presented-on average-negative pretest-to-sustained-effect change values. 
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An assessment of individual pupil change showed that at least 50% of the pupils in 14 of the 16 study 
groups had positive pretest-to-sustained effect change values. Served pupils in Math-Basic and Math- 
Advanced fell beicw the 50% level. Of pupils in the Reading-Basic exited group, 79.5% had positive 
pretest-toiX)sttest change values. 



Sumrnary 

In June 1992, the Colunft)us Public Schools submitted a similar Sustained Ejects Study, The current 
study was conducted using the same methodology~for the most part-to that used to report findings from 
the 1989-91 study. Recommendations from that study could be postulated for the current study: 

1 . Exited groups, by definition and by observation, will gain more NCEs during their year of exposure 
to Chapter 1 programming than will any of the remaining groups. 

2. Exited groups will lose wore NCEs during year two than will any of the remaining groups, 

3. Evaluation groups, again, by definition and observation, will have tower "long-term" (pretest to 
sustained effect) gain than will be observed for served groups. 
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Elementary and Secondary Education Act - Chapter 1 



REPORT OF THE CHAPTER 1 
SUSTAINED EFFECTS STUDY 

1992-94 

Introduction 



ESEA Chapter 1 evaluation guidelines require school districts receiving Chapter 1 funds to conduct a 
study of the sustained effects (SE) of Chapter 1 progranfis. The study is to be conducted once every three 
years and the results are to be used in the planning of Chapter 1 subsequent progranns. 

This study exairined the effect of Chapter 1 service on four skill areas: Total Reading (TR, basic), 
Reading Comprehension (RC, advanced), Total Math (TM, basic), and Math Concepts and Applications 
(MA. advanced). This examination used a Spring - Spring - Spring testing design: pretest 4/92. posttest 
4/93. and sustained effects 4/94. Prirriary emphasis was given to detemiining the extent to which the 
reading and math effects obtained by pupils in the 1992-93 Chapter 1 Evaluation Sanple (i«e., study- 
eligible pupils) were sustained during the 1993-94 school year. Use of this design made possible 
identification of Chapter 1 pupils in temis of pretest/posttest change, posttest/sustained effects change, and 
pretest/sustained effects change. 

Within each of the four skill areas of Chapter 1 instruction, four groups of study-eligible pupils were 
evaluated: not served by Chapter 1 in 1993-94 (exited pupils); in the 1993-94 evaluation sample; served 
but not in the 1993-94 evaluation sample; and total (sum of the first three groups). Results are presented 
by 1992-93 grade levels (grades 2-7). 

In this context, this study addressed three research questtons: 

a. What effect did service have on achievement level pretest (4/92) to posttest (4/93)? 

b. What achievement-level sustained effect was observed posttest (4/93) to sustained effect (4/94)? 

c. What net-effect achievement level was observed pretest (4/92) to sustained effect (4/94)? 



Method 

Study Eligibility 

Five criteria were used to select 1992-93 Chapter 1 pupils for analysis. To be included in the study, a 
pupil had to meet each of these five criteria: 

1 . The pupil was served in either a Chapter 1 reading or mathematics program in a Columbus Public School 
during the 1992-93 school year Nonpublic program pupils were not included in the study due to the 
small number of these pupils and the fact that their test results indicated that, as a group, they are 
rK>t typical of the public school group. 
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2. The pupil was in grades 2-7 during the 1992-93 school year. Pupils in grade 1 were not included 
because they were not tested on the requisite 12-nx)mh basis. Pupils in grade 8 were not included 
in the study because it was not feasible to collect sustained effects test scores in the ninth grade. 
This was because Chapter 1 services were not provided beyond grade 8 and no achievement 
testing was conducted by the district at grades 9 and above. 

3. The pupil was in the 1992-93 evaluation sample by virtue of having valid pretest/posttest scores and 
meetir^g one of the following attendance criteria: for all reading programs, (see Table 2), attended 
at least 80% of the instructional days or was discontinued; for n^th progranis. attended at least 
50% of the instructional days or was discontinued. A pupil could be in the basic skills sanpte. the 
advanced skills sample, or both. 

4. The pupil was promoted to the next grade for the 1993-94 school year. This foltows Chapterl evaluation 
procedure for pretest/posttest analysis and allows for a more meaningful interpretation of results. 

5. The pupil had test scores for all three testing periods (Spring 1 992, Spring 1 993. Spring 1 994). 

The data file for 1992-93 Chapter 1 servrce (see eligibility criteria #1-3) contained a total of 4.031 
potential cases for the study. Application of the five selection criteria produced the included pupils 
displayed in Table 1. The four observed groups are arrived to show nr^embership by instructtonal area 
(reading or math) and skills area (basic or advanced). For example, the Table 1 "Total" row shows the 
number of pupils per instructional and skill area. There were 1.425 pupils who were selected by the five 
criteria for the Reading-Basic group. There were 577 pupils who were in the exited group, Reading-Basic. 
The test series and subtests that were used to measure basic and advanced skills are given in the next 
section (see Table 3). 

Table 2 shows the mrvber of potential pupils for this study by 1992-93 program name and actual 
1993-94 service project name. Counts for 1993-94 do not include exited pupils. 

Test Instmments 

Chapter 1 pupils were administered test instmments as described in Table 3. For sustained effects 
(SE) values, all pupils took the grade and time-of-year appropriate tests of the California Achievement Test 
(CAT: CTB/McGraw-Hill. 1985). The 1992-93 second and third graders took the grade and tirDe-of-year 
apprx)priate tests of the Metropolitan Achievement Tests (MAT6: The Psychotogk^al Corporatton. 1985. 6th 
ed.) as a pretest and posttest. However, these pupils took a CAT as their sustained effects measurement 
in April 1994. 

Subtests used to measure the basic and advanced skills areas for grades 3-7 are as foltows: 
Reading-Basic (RB) = Total Reading (Reading Vocabulary and Reading Comprehenston); Reading- 
Advanced (RA) = Reading Comprehenston; Math-Basfc (MB) = Total Math (Math Computatton and Math 
Concepts and Applications); Math-Advanced (MA) = Math Concepts and Applications. For the MAT6 used 
at grades 2 and 3. Word Recognitton Skills, Reading Vocabulary and Reading Comprehenston make up 
Total Reading. 

Procedure 

Three data files were analyzed in this study. Two of these were compensatory education Program 
Involvement RIes (PIF: 1992-93, PIF: 1993-94). The third data file used was the April 1994 districtwkJe 
testing file (DW) maintained by the Columbus Publrc Schools. 

The 1992-93 PIF file was the basis for detemiining the pupils who met the first three eligibility criteria 
listed in the Method section. Records for pupils who met the first three eligibility criteria were matched by 
student nutvber to the 1993-94 PIF file to detemiine Chapter 1 servfce status and grade prorTX)tk)n status. 
Then, records for these pupils were matched by student number to the DW file to secure SE values! 
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Table 1 

Sustained Effecte Study 
Pupil Counts by Observed Group and Skills Area 
After Selection Criteria were Applied 



Skills Area 



Observed Reading Math 



Groups 


Basic 


Advanced 


Basic 


Advanced 


Exited 


577 


590 


964 


983 


I.T 1993-94 Evaluation Sample 


780 


810 


681 


683 


Served (not in sample) 


68 


66 


106 


124 


Total 


1.425 


1.466 


1.751 


1.790 



^Group definition: E^dlfid. - in the 1 992-93 Evaluation Sample but not served in the 1 993-94 program. 



Evaluation Sample - in the 1992-93 Evaluation Sample and in the 1993-94 Evaluation 
Sample» 

Served - in the 1992-93 Evaluation Sample, served in 1993-94, but not in the 1993-94 
Evaluation Sample. 

Mai - the sum of Exited, Evaluation Sample and Served. 



Table 2 

Sustained Effects Study 
Pupil Counts by Year and Project Code 
for Pupils Meeting All Five Selection Criteria 



Year 



1992-93 



Program Name 


Code 


Grades 


Count 


Reading Elementary 


93004 


2-5 


1,304 


Reading Middle 


93014 


6-7 


423 


Eariy Literacy 


93398 


2 


94 


Mathematics Elementary 


93005 


3-5 


1.298 


Mathiematics Middle 


93015 


6-7 


912 


Total 






4.031 



1993-943 



Program Nama 


Code 


Grades 


Count 


Reading Elementary 


94004 


2-5 


694 


Reading Middle 


94014 


6-7 


337 


Early Literacy 


94398 


2 


2 


Mathematics Elementary 


94005 


3-5 


331 


Mathematics Middle 


94015 


6-7 


649 








2,013 



^Given membership in both years. 
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Tables 



Sustained Effects Study 
Test Level arid Form 



1992-93 


1993-94 


4-92 


4-93 


4-94 


Treatment 


Sustained Effect 


Pretest 


Posttest 


Sustained Effect 


Grade 


Grade 


Test/Level/Fonn 


Test/Level/Fonn 


Test/Level/Fonn 


2 


3 


?^^T6 Primer L 


MAT6 Primary 1 L 


CAT 13 E 


3 


4 


^MT6 Primary 1 L 


MAT6 Primary 2 L 


CAT 14 E 


4 


5 


CAT 13 E 


CAT 14 E 


CAT 15 E 


5 


6 


CAT 14 E 


CAT 15 E 


CAT 16 E 


6 


7 


CAT 15 E 


CAT 16 E 


CAT 17 E 


7 


8 


CAT 16 E 


CAT 17 E 


CAT 18 E 



Group 


N 


EXITRB 


577 


EVSAMPRB 


780 


SERVEDRB 


68 


TOTALRB 


1,425 


EXITRA 


590 


EVSAMPRA 


810 


SERVEDRA 


66 


TOTALRA 


1.466 


EXITMB 


964 


EVSAMPMB 


681 


SERVEDMB 


106 


TOTALMB 


1.751 


EXITMA 


983 


EVSAMPMA 


683 


SERVEDMA 


124 


TOTALMA 


1.790 



Table 4 

Sustained Effects Study 
Analysis Groups^ 



Description 



Exited, Reading- Basic (RB) 

Evaluation Sample, Reading-Basic (RB) 

Served-not-in-Sample, Reading-Basic (RB) 

Sum of EXITRB. EVSAMPRB and SERVEDRB (RB) 

Exited. Reading-Advanced (RA) 

Evaluation Sample. Reading-Advanced (RA) 

Sen/ed-not-in-Sample. Reading-Advanced (RA) 

Sum of EXITRA, EVSAMPRA and SERVEDRA (RA) 

Exited, Math-Basic (MB) 

Evaluation Sample, Math-Basic (MB) 

Sen/ed-not-in-Sample, Math-Basic (MB) 

Sum of L-XITMB, EVSAMPMB and SERVEDMB (MB) 

Exited, Math-Advanced (MA) 

Evaluation Sample, Math-Advanced (MA) 

Served-not-in-Sample, Math-Advanced (MA) 

Sum of EXITMA, EVSAMPMA and SERVEDMA (MA) 



^ Math Basic and Math Advanced were not tested at grade 2. 
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By matching the files this way, it was possible to identify and keep for further analysis only those pupil 
records who met all fi^'e eligibility criteria. The resulting subset of 1992-93 records were partittoned into 16 
separate datasets according to group membership (see Table 4), 

All data analyses were conducted using software code written in SAS (Statistical Analysis System, 
V.6.08), Each of the sixteen groups was processed individually to produce graphs and tables for each 
grade level Involved in the study. For example, Figure 1, "Reading-Basic (Total Reading)", uses gr^je 2 
data from Tables 5-8. Figures 1-6 use data from Tables 5-8; Figures 7-12 use data from Tables 9-12; 
Figures 13-18 use data from Tables 1 3-1 6; and Figures 1 9-22 use data from Tables 1 7-20. 

Pupil Counts. Each of the 16 major datasets consists of at least 50 pupils. However, for example, when 
the "sen/ed" group for Reading-Basic is split by grade level, pupil count becomes less than 10 in certain 
instances. Conclusions made based on figures foi these low-count subsets should be tempered 
accordingly. 

Test/Level/Fomn Considerations. Second and third-grade pupils were tested with MAT6, with sustained 
effect'3 scores from CAT. Therefore, extreme caution should be used in interpreting results. Conplete 
test/levei/form infonnation is contained in Table 3. 

The following figures (1-22) and tables (5-20) report grade level findings. Each figure shows four 
cun/es: Exited, Evaluation Sample, Sen/ed-not-in-Sample, and Total. The legend below each figure 
associates the symbol used at each measurement point with a particular cun/e on the graph. Tabular data 
by the same skill areas is associated by name with each specific figure. 

Change values are presented in the nan-ative betow each figure. Pretest/posttest change and 
posttest/sustained effects change are reported. Tabularly, posttest minus pretest change is labeled "POS- 
PRE". Sustained effect minus posttest is represented by "SE-POS". Sustained effect minus pretest is 
represented by "SE-PRE". 

Posttest-Pretest. Using the 4/93 value as posttest and the 4/92 value as pretest, the average change value 
was computed for each of the 16 major groups in the study. Table 21 shows this •Ireatment" effect. All four 
exited groups, as expected, had change values-ix>sitive-greater than 10 NCE points, as did Total Math- 
Basic for ail three groups combined (TOTALMB). 

Evaluation sample groups had modest gains (RA, MB, MA) or negative values (RB). Served groups 
all had positive change values. These values were, unifomily, nDore positive than those for evaluation 
sample groups. 

Total group {vnosX influenced by the exited group) change values are al! positive, but only the 
TOTALMB group approximates change as was observed for the exited groups. 

Sustained Effect-Posttest. Short-temn sustained effect (SE) change was evaluated by using the 4/94 SE 
value minus the 4/93 posttest value. Table 22 shows this "short-temi" retention effect. All four exit groups 
have negative change values, the most significant being the MB -8.0; i.e., from posttest to SE 
measurement about 8 NCE points were lost with respect to Math-Basic for the exit group. 

Evaluation sample groups all had positive change values (i.e., relatively, these groups "grew" during 
the second year of treatnrient). The same was not tme for all four served groups. Only SERVEDRB 
"change" is positive; SERVEDMB lost more than 3 NCEs. Regarding total groups, all four had negative 
change values, due to the exited group influence. 
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Sustained Effect-Pretest. Table 23 presents the results for "long-term" retention effect, where the change 
value was calculated by subtracting the 4/92 pretest value from the 4/94 sustained effect value. Most 
calculated change values reported in Table 23 are positive; i.e.. regardless of study group membership, 
across the two-year interval NCE change was posStrve. Only SERVEDMB and SERVEDMA have negative 
change values. 

The exited groups had the highest change values: RB = 10.6, RA = 10.8, MB = 97, and MA = 8.1. 
For the two reading skill areas, sen/ed groups had larger change values than did evaluation sample groups. 
The opposite was true for the math skill areas. 

Individual Pupil Change 

Of interest is the percent of each of the 16 study groups which had positive change values for each of 
the three major research questfons. Table 24 is a summary of this information. 

From this table it is evident that "exit" groups are rrost likely to show severe drop-off. comparing 
posttest-pretest to sustained effect-posttest change. For example, the exited group in the Math-Basic skill 
area for posttest-pretest showed 95.7% of the pupils with pcf^itive change. But for sustained effect-posttest, 
this value dropped to 25.4% positive change. Or, of the 964 pupils in this group, only about 245 scored 
higher on the 4/94 measurerront than on the 4/93 measurement. 

In contrast, the evaluation sample groups increased the percent in three of the four skill areas. The 
sen/ed groups decreased for all four groups. 

Positive change percents for sustained effects-pretest observations range from a k)w of 44.4 
(SERVEDMB) to a high of 79.5 {EXI f EDRB). 

Discussion 



Application of the five eligibility criteria to the 1992-93 PIF dataset {N=4,031) produced an analysis 
dataset with 1 ,936 reading cases and 2,542 math cases. Thus, all analysis cases were: (a) on the 1992- 
93 PIF file; (b) in grades 2-7 during the 1992-93 school year; (c) in the 1992-93 evaluation sample; (d) 
proi7X)ted to the next grade for the 1993-94 school year; and (e) tested in all three measurement periods 
{4/92, 4/93, and 4/94). 

Total" statistics reported are weighted heavily toward "evaluation" group membership for the skill area 
Reading: Nrb=780; Nra=810 (see Table 4). 

Total statistics reported for Math are-in contrast-tTK)st heavily weighted by "exit" group membership; 

NmB=964; NmA=983. Recognizing that exited groups do not receive second-year treatment, it is not 
surprising that "total" groups Math-Basic and -Advanced show a second-year NCE decline similar to that 
obsen/ed for exited groups, nor is it remart^able that exit groups-in general-themsetves exhibit a mari<ed 
second-year decline. 

Exited groups, by definition, had the highest posttest-pretest positive change, greater than 10 NCE 
units in all cases: RB = 14.0; RA = 16.1; MB = 17.7; MA = 13.7. Exited groups also had the most 
negative sustained effect-posttest change: RB = -3.4; RA = -5.4; MB = -8.0; MA = -5.6; e.g., about 45% 
of the MB change pretest to posttest disappeared posttest to sustained effect. 
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On the other hand, evaluation sample groups showed positive change during the second year of 
treatment. Although change for these groups is not as impressive as for exited groups, positive change 
does appear to be realistic and particularly likely during the secorKl treatment year. 

The comparison of pretest and sustained-effect change reinforces this notion. The net effect of 
treatment across two school years is positive for 15 of the 16 analysis groups (see Table 23; sustained 
effects scores are-on average-higher than pretest scores). Again, exited groups have the higher change 
values but positive effect is evident for all groups except for senved Math-Basic. Consider that pupils who 
were selected for sen/ice in the second year of the study scored low on the posttest for the previous year. 
First-year change (pretest to posttest) showed a negative value for the evaluation sample group in 
Reading-Basic (-0.4). 

Regarding change at the individual pupil level, pretest to sustained effect, more than one-half of all 
pupils had change values greater than zero for 1 5 of the 1 6 analysis gnoupj^. Over 79% of the exited group 
in Reading-Basic pupils increased in NCE from pretest to sustained effect. And, over 77% of the exited 
group in Math-Basic had positive change values. 

Summary 



The Colunnbus Public Schools completed the sustained-effects study in accordance with the Chapter 
1 federal guidelines. The result of this study will be used in planning future Chapter 1 programs. 

In June 1 992, the Columbus Public Schools submitted a similar Sustained Effects Study. The current 
study was conducted using the sarro methodology-for the nfx>st part-to that used to report findings from 
the 1 989-91 study. Recommendations from that study could be postulated for the current study: 

1. Exited groups, by definition and by obsen/ation, will gain more NCEs during their year of exposure 
to Chapter 1 programming than will any of the remaining other groups. 

2. Exited groups will lose more NCEs during year two than will any of the remaining groups, 

3. Evaluation groups, again, by definition and obsen/ation, will have lower "long-term" (pretest to 
sustained effect) gain than will be obsen/ed for sen/ed groups. 



ERLC 
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SUSTAINED EFFECTS STUDY 
Choptai' 1 NCK Moan Values by Teat Date 

READiNQ - BASIC (IMftl RMdln0) 

Orade 2 




4/92 4/93 4/94 

Date of Testing 



L£CEMO: 1993-94 EiHad-Progrom Group 0- 1993-94 Evoluob'^ Sompla Group A- 1993-94 Sirved-not-in-Sompl« Group • • Tolol Cojwt 



Figure 1. Average NCE for Total Reading pretest (4/92), posttest (4/93), and sustained effects (4/94) 
scores for second-traders in the 1992-93 Chapter 1 Evaluation Sample. 

The average pretestix)sttest change in Total Reading for the four grxxjps of pupils was: exited 1 9.0 (171), 
evaluation sample -2.0 (225), sen/ed 1.1 (27). and total 6.7 (423). The average posttest- sustained e'fects 
change for the four groups of pupils was: exited -3.1 (171), evaluation sample 4.8 (225), sen/ed 0.9 (27), 
and total 1.4 (423). The number of pupils appears in parentheses. 
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Figure 2. Average NCE for Total Reading pretest (4/92), posttest (4/93). and sustained effects (4/94) 
scores for third-graders in the 1992-93 Chapter 1 Evaluation Sample. 

The average pretest-posttest change in Total Reading for the four groups of pupils was: exited 1 1 .7 (146). 
evaluation sample -1.4 (176), served 1.3 (16), and total 4.4 (338). The average posttest-sustained effects 
change for the four groups of pupils was: exited -0.2 (146), evaluation sanple 4.9 (176), served 6.7 (16), 
arxJ total 2.8 (338). The number of pupils appears in parentheses. 
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Figure 3. Average NCE for Total Reading pretest (4/92). posttest (4/93), and sustained effects (4/94) 
scores for fourth-graders in the 1992-93 Chapter 1 Evaluation Sample. 

The average pretest-posttest change in Total Reading for the four groups of pupils was: exited 12.4 (70), 
evaluation sample 0.7 (108), served 2.9 (8), and total 5.2 (186). The average posttest-sustained effects 
change for the four groups of pupils was: exited -3.8 (70), evaluation sample 4.2 (108), sen/ed 7.4 (8), and 
total 1 .3 (186). The number of pupils appears in parentheses. 
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Figure 4. Average NCE for Total Reading pretest (4/92). posttest (4/93), and sustained effects (4/94) 
scores for fifth-graders in the 1992-93 Chapter 1 Evaluation Sarrple. 



The average pretest i^osttest change in Total Readirfg for the four groups of pupils was: exited 12,5 (66), 
evaluation sample 4.1 (71), served 6.0 (7). and total 8.0 (144). The average posttest-sustained effects 
change for the four groups of pupils was: exited -6.7 (66), evaluation sample -1 .9 (71), served -4.1 (7), and 
total -42 (144), The number of pupils appears in parentheses. 
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Figure 5. Average NCE for Total Reading pretest (4/92), posttest (4/93), and sustained effects (4/94) 
scores for sixth-traders in the 1992-b,'^ Chapter 1 Evaluation Sample. 



The average pretest-posttest change in Total Reading for the four groups of pupils was: exited 7.6 (47), 
evaluation sample -0.9 (106), served 2.0 (5), and total 1.7 (158). The average posttest-sustained effects 
change for the four groups ot' pupils was: exited -3.6 (47), evaluation sample 1 .3 (1 06). served 1 .6 (5). and 
total -0.1 (1 58). The number of pupils appears in parentheses. 
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Figure 6. Average NCE for Total Reading pretest (4/92). posttest (4/93), arxJ sustained effects (4/94) 
scores for seventh-graders in the 1992-93 Chapter 1 Evaluation Sarr^Ie. 

The average pretest-posttest change in Total Reading for the four groups of pupils was: exited 13.5 (77), 
evaluation sample 1.0 (94), served 7.2 (5). and total 6.7 (176). The average posttest-sustained effects 
change for the four groups of pupils was: exited -7.0 (77), evaluation sample 0.2 (94). sen/ed -7.0 (5). and 
total -3.2 (176). The number of pupils appears in parentheses. 
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Figure 7. Average NCE for Reading Coirprehension pretest (4/92), posttest (4/93), and sustained effects 
(4/94) scores for second-graders in the 1992-93 Chapter 1 Evaluation Sample. 



The average pretest-posttest change in Reading Comprehension for the four groups of pupils was: exHed 
22.9 (176), evaluation sanpie 1.8 (231), sen/ed 7.5 (26), and total 10.7 (433). The average posttest- 
sustained effects change for the four groups of pupils was: exHed -5.8 (176). evaluation sample 3.4 (231), 
sen/ed 0.0 (26), and total -0.6 (433). The number of pupils appears in parentheses. 



ERIC 



P:\P530NSUERPT95 
2.22-95 12^7 PM 



IB 



18 



SUSTAINED EFFECTS STUDY 

Chapter 1 NCE Mean Values by Teat Date 
READINa - ADVANCED - (RMd!n0 Compr«h«nslon) 

Orade 3 




4/92 4/93 4/94 

Date of Teatin^^ 



LtGENO: □« 1993-9* Eti'Ud-Progrom Croup 0 «• '993-9* Evduotlon Sompla Croup A « 1 993-9* 5ervad-not-in-Somp<e Group • « Toto* Cosa 



Figures. Average NCE for Reading Comprehension pretest (4/92). posttest (4/93), and sustained effects 
(4/94) scores for third-graders in the 1992-93 Chapter 1 Evaluation Sample. 

Tha average pretestix>sttest change in Reading Conprehension for the four groups of pupils was: exited 
13.2 (152), evaluation sample -3.9 (197), served 2.7 (15), and total 3.5 (364). The average posttest- 
sustained effects change for the four groups of pupils was: exited -3.3 (152), evaluation sample 4.5 (197), 
served 2.8 (15). and total 1 .2 (364). The number of pupils appears in parentheses. 
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Rgure 9. Average NCE for Reading Comprehension pretest (4/92), posttest (4/93), and sustained effects 
(4/94) scores for fourth-graders in the 1992-93 Chapter 1 Evaluation Sample. 

The average pretest-posttest change in Reading Comprehension for the four groups of pupils was: exited 
14.1 (71), evaluation sample -0.6 (108). served 6.4 (8). and total 5.3 (187). The average posttest-sustained 
effects change for the four groups of pupils was: exited -5.3 (71), evaluation sample 5.0 (108), served 6.3 
(8), and total 1 .1 (187). The number of pupils appears in parentheses. 
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Rgure 10. Average NCE for Reading Comprehension pretest (4/92). posttest (4/93). and sustained effects 
(4/94) scx5res for fifth-traders in the 1992-93 Chapter 1 Evaluation Sanple. 

The average pretest-posttest change in Reading Comprehension for the four groups of pupils was: exited 
12.7 (67), evaluation sample 4.4 (72), served 7.4 (7), and total 8.4 (146). The average posttest-sustained 
effects change for the four groups of pupils was: exited -5.6 (67), evaluation sanple -0.5 (72), sen/ed -6.1 
(7), and total -3.1 (146). The number of pupils appears in parentheses. 
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Figure 1 1 . Average NCE for Reading Comprehension pretest (4/92). posttest (4/93), and sustained effects 
(4/94) scores for sixth-graders in the 1992-93 Chapter 1 Evatuatfon Sample. 



The average pretest-posttest change in Reading Comprehension for the four groups of pupils was: exited 
8.9 (47). evaluation sample -0.3 (107). sen/ed 7.2 (5). and total 2.7 (159). The average posttest-sustained 
effects change for the four groups of pupils was: exited -4.3 (47), evaluation sanple 3.9 107). served -3.4 
(5), and total 1 .2 (159). The number of pupils appears in parentheses. 
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Figure 12. Average NCE for Reading Connprehension pretest (4/92), posttest (4/93), and sustained effects 
(4/94) scores for seventh-graders in the 1992-93 Chapter 1 Evaluation Sanple. 

The average pretestiwsttest change in Reading Comprehension for the four gtxxjps of pupils was: exited 
15.6 (77), evaluation sample 4.0 (95), served 7.8 (5), and total 9.1 (177). TTie average posttest-sustained 
effects change for the four groups of pupils was: exited -9.0 (77), evaluation sanple -2.8 (95), served -1 1 .8 
(5), and total -5.8 (1 77). The number of pupils appears in parentheses. 
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Rgure 13. Average NCE for Total Math pretest (4/92), posttest (4/93), and sustained effects (4/94) scores 
for third-graders in the 1992-93 Chapter 1 Evaluation Sample. 

The average pretestiX)Sttest change in Total Math for the four groups of pupils was: exrted 21,4 (198), 
evaluation sample 4.9 (1 28), served 1 0.9 (1 0), and total 1 4.8 (336). The average posttest-sustained effects 
change for the four groups of pupils was: exited -7.8 (198), evaluation sample -6.7 (128), served -4.7 (10). 
and total -2.2 (336). The number of pupils appears in parentheses. 
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Rgure 14. Average NCE for Total Math pretest (4/92), posttest (4/93), and sustained effects (4/94) scores 
for foorth-Qraders in the 1992-93 Chapter 1 Evaluation Sample. 



The average pretestiX)Sttest change in Total Math for the four groups of pupils was: exited 17.0 (228), 
evaluation sample 3.4 (141), served 3.3 (8), and total 1 1 .6 (377). The average posttest-sustained effects 
change for the four grxxips of pupils was: exited -5.9 (228), evaluation sample 3.6 (141). ser'ed 5.4 (8). 
and total -2.1 (377). The number of pupils appears in parentheses. 
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Figure 15. Average NCE for Total Math pretest (4/92), posttest (4/93). and sustained effects (4/94) scores 
for fifth-graders in the 1992-93 Chapter 1 Evaluation Sanrple. 

The average pretestiX)sttest change in Total Math for the four groups of pupils was: exited 17.9 (208), 
evaluation sample ^).9 (136), served 1.0 (33), and total 10.3 (377). The average posttest-sustained effects 
change for the four groups of pupils was: exited -10.1 (208), evaluation sanple 0.8 (136). served -3.9 (33), 
and total -5.6 (377). The number of pupils appears in parentheses. 
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Rgure 16. Average NCE for Total Math pretest (4/92), posttest (4/93). and sustained effects (4/94) scores 
for sixth-graders in the 1992-93 Chapter 1 Evaluation Sample. 

The average pretestiX)sttest change in Total Math for the four groups of pupils was: exited 15.1 (174), 
evaluation sample 0.8 (153), sen/ed 0.0 (15), and total 8.0 (342). The average posttest-sustained effects 
change for the four groups of pupils was: exited -6.2 (174), evaluation sample 0.7 (153). served -2.9 (15). 
and total -3.0 (342). The number of pupils appears in parentheses. 
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Rgure 17. Average NCE for Total Math pretest (4/92), posttest (4/93), and sustained effects (4/94) scores 
for seventh-graders in the 1992-93 Chapter 1 Evaluation Saniple. 



The average pretest iX)Sttest change in Total Math for the four groups of pupils was: exited 16.3 (156). 
evaluation sample 1.2 (123), sen/ed 3.6 (40). and total 8.9 (319). The average posttest-sustained effects 
change for the four groups of pupils was: exited -10.6 (156), evaluation sanrple -2.6 (123). served -4.1 (40), 
and total -6.7 (319). The number of pupils appears in parentheses. 
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Rgure 18. Average NCE for Concepts and Applications pretest (4/92), posttest (4/93), and sustained 
effects (4/94) scores for third-graders in the 1992-93 Chapter 1 Evaluation Sample. 

The average pretestix)$ttest change in Concepts and Applications for the four groups of pupils was: exHed 
12.0 (202), evaluation sample -0.1 (129). served 9.9 (15), and total 7.4 (346). The average posttest- 
sustained effects change for the four groups of pupils was: exited -€.7 (202), evaluation sanrple 2.2 (129), 
served -1 2.7 (1 5), and total -3.7 (346). The number of pupils appears in parentheses. 
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Figure 19. Average NCE for Concepts and Applications pretest (4/92), posttest (4/93), and sustained 
effects (4/94) scores for fourth-graders in the 1992-93 Chapter 1 Evaluation Sample. 

The average pretest-posttest change in Concepts and Applications for the four groups of pupils was: exited 
12.9 (228), evaluation sample 1.1 (142), sen/ed 4.9 (10). and total 8.3 (380). The average posttest- 
suslained effects change for the four groups of pupils was: exited -3.7 (228), evaluation sanple 4.1 (142), 
served 0.9 (10), and total -0.7 (380). The number of pupils appears in parentheses. 
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Rgure 20. Average NCE for Concepts and Applications pretest (4/92), posttest (4/93), and sustained 
effects (4/94) scores for fifth-graders In the 1992-93 Chapter 1 Evaluation Sample. 



The average pretest-posttest change in CJoncepts and Applications for the four groups of pupils was: exited 
17.1 (213), evaluation sample 0.5 (136), served 3.5 (35), and total 10.0 (384). The average posttest- 
sustained effects change for the four groups of pupils was: exited -7.1 (213), evaluation sample 2.3 (136), 
served -3.6 (35). and total -3.4 (384). The number of pupils appears in parentheses. 
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Figure 21 . Average NCE for Concepts and Applications pretest (4/92), posttest (4/93), and sustained 
effects (4/94) scores for sixth^raders in the 1992-93 Chapter 1 Evaluation Sample. 



The average pretest-posttest change in Concepts and Applications for the four groups of pupils was: exited 
11.5 (180), evaluation sample -0.3 (153), served -5.4 (20), and total 5.4 (353). The average posttest- 
sustained effects change for the four groups of pupils was: exited -3,5 (180), evaluation sanple 2.7 (153), 
seived 1 .3 (20), and total -0.6 (353). The number of pupils appears in parentheses. 
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Figure 22. Average NCE for Concepts and Applications pretest (4/92), posttest (4/93). and sustained 
effects (4/94) scores for seventh-graders in the 1992-93 Chapter 1 Evaluation Sample. 



The average pretest-posttest change in Concepts and Applications for the four groups of pupils was : exited 
14.9 (160). evaluation sample 0.9 (123). sen/ed 1.8 (44). and total 7.9 (327). The average posttest- 
sustained effects change for the four groups of pupils was: exited 7.4 (1 60). evaluation sanple -0.5 (1 23). 
served -1 .7 (44). and total -4.0 (327). The number of pupils appears in parentheses. 
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Table 21 



Sustained Effects Study 
Treatment Change Analysis^ 







4/93 


4/92 




Study Group 


N 


Posttest 


Pretest 


Change'^ 


r* VI TDD 

CAII KB 


577 


39.1 


25.1 


14.0 


EVSAMPRB 


780 


23.4 


23.8 


-0.4 


o r~ p>v /r~ r\ p> o 

SERVEDRB 


68 


25.5 


23.1 


2.4 


TUTALRB 


1,425 


29.8 


24.3 


5.5 


cXITRA 


590 


42.6 


26.5 


16.1 


tvoAMrHA 


610 


26.7 


26.4 


0.3 


ohRVcDRA 


66 


29.7 


23.4 


6.3 


T/^TAI DA 


1,466 


33.2 


26.3 


6.9 


EXITMB 


964 


46.5 


28.9 


17.7 


EVSAMPMB 


681 


29.7 


27.6 


2.2 


SERVEDMB 


106 


29.6 


26.6 


2.9 


TOTALMB 


1.751 


39.0 


28.2 


10.7 


EXITMA 


983 


46.1 


32.4 


13.7 


EVSAMPMA 


683 


31.8 


31.4 


0.4 


SERVEDMA 


124 


32.1 


29.7 


2.3 


TOTALMA 


1.790 


39.6 


31.8 


7.8 



^4/93 Posttest Mean minus 4 / 92 Pretest Mean. 

b Rounding caused some change values to differ from expected values. 
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Table 22 



Sustained Effects Study 
Short-Term Sustained Effect Change Analysis^ 







4/94 


4/93 




Study Group 


N 


SETest 


Posttest 


Change*^ 


CVIXDD 

bXI 1 Ho 


577 


35.7 


39.1 


-3.4 


bVoAMPHD 


780 


26.5 


23.4 


3.1 


ohRVcL/KD 


68 


27.4 


25.5 


2.0 


Tr\TA 1 DD 

1 U I ALHd 


1,425 


30.2 


29.8 


-0.4 


C VIXD A 

hXI IKA 


590 


37.2 


42.6 


-5.4 


hVoAMrKA 


810 


29.5 


26.7 


2.9 


ohRVhUKA 


66 


29.3 


29.7 


-0.9 


Xl^XAl OA 

lUI ALHA 


1,466 


32.6 


33.2 


-0.6 


EXITMB 


964 


38.5 


46.5 


-8.0 


EVSAMPMB 


681 


31.6 


29.7 


1.9 


SERVEDMB 


106 


26.3 


29.6 


-3.2 


TOTALMB 


1.751 


35.1 


39.0 


-3.9 


EXITMA 


983 


40.5 


46.1 


-5.6 


EVSAMPMA 


683 


34.0 


31.8 


2.2 


SERVEDMA 


124 


29.2 


32.1 


-2.9 


TOTALMA 


1.790 


37.2 


39.6 


-2.4 



^4/94 Sustained Effect Mean minus 4 / 93 Posttest Mean. 

b Rounding caused some change values to differ from expected values. 



P:\P530\SUERPT95 
2-27.95 1 1:40 AM 



52 



Table 23 



Sustained Effects Study 
Long-Term Sustained Effect Change Analysis^ 







4/94 


4/92 




Study Group 


N 


SETest 


Pretest 


Change*' 


EXITRB 


577 


35.7 


25.1 


10.6 


EVSAMPRB 


780 


26.5 


23.8 


2.7 


SERVEDRB 


68 


27.4 


23.1 


4.4 


TOTALRB 


1,425 


30.2 


24.3 


5.9 


EXITRA 


590 


37.2 


26.5 


10.8 


EVSAMPRA 


810 


29.5 


26.4 


3.2 


SERVEDRA 


66 


29.3 


23.4 


5.9 


At A 

TOTAL RA 


1,466 


32.6 


26.3 


6.3 


EXITMB 


964 


38.5 


28.9 


9.7 


EVSAMPMB 


681 


31.6 


27.6 


4.0 


SERVEDMB 


106 


26.3 


26.6 


-0.3 


TOTALMB 


1,751 


35.1 


28.2 


6.9 


EXITMA 


983 


40.5 


32.4 


8.1 


EVSAMPMA 


683 


34.0 


31.4 


2.6 


SERVEDMA 


124 


29.2 


29.7 


-0.5 


TOTALMA 


1.790 


37.2 


31.8 


5.4 



^4/94 Sustained Effect Mean minus 4 / 92 Pretest Mean. 

b Rounding caused some change values to differ from expected values. 
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Table 24 



Sustained Effects Study 
Individual Pupil Change 
Percent Greater Than Zero® 



Study Group 


N 


Posttest 
minus 
Pretest 


Sustained Effect 
minus 
Posttest 


Sustained Effect 
minus 
Pretest 


EXITRB 


577 


91.9 


35.9 


79.5 


EVSAMPRB 


780 


45.6 


60.4 


58.3 


SERVEDRB 


68 


55.9 


48.5 


61.8 


TOTALRB 


1.425 


64.8 


49.9 


67.1 


EXITRA 


590 


89.8 


35.9 


75.8 


EVSAMPRA 


810 


47.9 


57.8 


57.7 


SERVEDRA 


66 


69.7 


48.5 


62.1 


TOTALRA 


1.466 


65.8 


48.6 


65.1 


EXITMB 


964 


95.7 


25.4 


77.5 


EVSAMPMB 


681 


57.7 


53.5 


58.3 


SERVEDMB 


106 


55.7 


38.7 


44.3 


TOTALMB 


1.751 


78.5 


37.1 


68.0 


EXITMA 


983 


88.9 


33.1 


74.4 


EVSAMPMA 


683 


48.6 


55.3 


57.8 


SERVEDMA 


124 


52.4 


41.9 


49.2 


TOTALMA 


1.790 


71.0 


42.2 


66.3 



^ The proportion of the group with a positive change value. 
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